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Abstract. We consider the inverse optimization problem associated with the 
polynomial program /* = min{/(x) : x G K} and a given current feasible 
solution y G K. We provide a systematic numerical scheme to compute an 
inverse optimal solution. That is, we compute a polynomial / (which may be 
of same degree as / if desired) with the following properties: (a) y is a global 
minimizer of / on K with a Putinar's certificate with an a priori degree bound 
d fixed, and (b), / minimizes ||/ — /|| (which can be the £i, £2 or £oo-norm 
of the coefficients) over all polynomials with such properties. Computing 
reduces to solving a semidefinite program whose optimal value also provides a 
bound on how far is /(y) from the unknown optimal value /*. The size of the 
semidefinite program can be adapted to computational capabilities available. 
Moreover, if one uses the £i-norm, then / takes a simple and explicit canonical 
form. Some variations are also discussed. 



1. Introduction 

Let P be the optimization problem /* = min {/(x) : x G K }, where 

(1.1) K:-{xe]R" : 5, (x) > 0, j - 1, . . . , m}, 

for some polynomials /, (gj) C M[x]. This framework is rather general as it encom- 
passes a large class of important optimization problems, including non convex and 
discrete optimization problems. 

Problem P is in general NP-hard and one is often satisfied with a local minimum 
which can be obtained by running some local minimization algorithm among those 
available in the literature. Typically in such algorithms, at a current iterate (i.e. 
some feasible solution y e K), one checks whether some optimality conditions (e.g. 
the Karush-Kuhn- Tucker (KKT) conditions) are satisfied within some e-tolerance. 
However, as already mentioned those conditions are only valid for a local minimum, 
and in fact, even only for a stationary point of the Lagrangian. Moreover, in many 
practical situations the criterion / to minimize is subject to modeling errors or 
is questionable. In such a situation, the practical meaning of a local (or global) 
minimum /* (and local (or global) minimizer) also becomes questionable. It could 
well be that the current solution y is in fact a global minimizer of an optimization 
problem P' with same feasible set as P but with a different criterion /. Therefore, 
if / is close enough to /, one might not be willing to spend an enormous computing 
time and effort to find the global (or even local) minimum /* because one might 
be already satisfied with the current iterate y as a global minimizer of P'. 

Inverse Optimization is precisely concerned with the above issue of determin- 
ing a criterion / as close to / as possible, and for which the current solution y is an 
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optimal solution of P' with this new criterion /. Pioneering work in Control dates 
back to Freeman and Kokotovic [8] for optimal stabilization. Whereas it was known 
that every value function of an optimal stabilization problem is also a Lyapunov 
function for the closed-loop system, in [8^ the authors show the converse, that is, 
every Lyapunov function for every stable closed-loop system is also a value function 
for a meaningful optimal stabilization problem. In optimization, pioneering works 
in this direction date back to Burton and Toint and 3^ for shortest path problems, 
and Zhang and Liu [501 [H], Huang and Liu [7 , and Ahuja and Orlin and [2 for lin- 
ear programs in the form minjc'x : Ax > b; r < x < s} (and with the £i-norm). 
For the latter, the inverse problem is again a linear program of the same form. 
Similar results also hold for inverse linear programs with the ^oo-norm as shown in 
Ahuja and Orlin [2^ while Zhang et al. [27 provide a column generation method for 
the inverse shortest path problem. In Heubergcr [6 the interested reader will find 
a nice survey on inverse optimization for linear programming and combinatorial 
optimization problems. For integer programming, Schaefer [16] characterizes the 
feasible set of cost vectors c e M" that are candidates for inverse optimality. It is 
the projection on ]R" of a (lifted) convex polytope obtained from the super-additive 
dual of integer programs. Unfortunately and as expected, the dimension of of the 
lifted polyhedron (before projection) is exponential in the input size of the problem. 
Finally, for linear programs Ahmed and Guan 1, have considered the variant called 
inverse optimal value problem in which one is interested in finding a linear criterion 
c S C C M" for which the optimal value is the closest to a desired specified value. 
Perhaps surprisingly, they proved that such a problem is NP-hard. 

As the reader may immediately guess, in inverse optimization the main difficulty 
lies in having a tractable characterization of global optimality for a given current 
point y G K and some candidate criterion /. This is why most of all the above cited 
works address linear programs or combinatorial optimization problems for which 
some characterization of global optimality is available and can be (sometimes) effec- 
tively used for practical computation. For instance, the characterization of global 
optimality for integer programs described in Schaefer jl6l is via the superadditive 
dual of Wolsey pJl §2] which is exponential in the problem size, and so prevents 
from its use in practice. 

This perhaps explains why inverse (non linear) optimization has not attracted 
much attention in the past, and it is a pity since inverse optimality could pro- 
vide an alternative stopping criterion at a feasible solution y obtained by a (local) 
optimization algorithm. 

The novelty of the present paper is to provide a systematic numerical scheme for 
computing an inverse optimal solution associated with the polynomial program P 
and a given feasible solution y S K. It consists of solving a semidefinite prograrrQ 
whose size can be adapted to the problem on hand, and so is tractable (at least 
for moderate size problems and possibly for larger size problems if sparsity is taken 
into account). Moreover, if one uses the ^i-norm then the inverse-optimal objective 
function exhibits a simple and remarkable canonical (and sparse) form. 



A semidefinite program is a convex (conic) optimization problem that can be solved efficiently. 
For instance, up to arbitrary (fixed) precision and using some interior point algorithms, it can be 
solved in time polynomial in the input size of the problem. For more details the interested reader 
is referred to e.g. Wolkowicz et al. |18) and the many references therein. 
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Contribution. In this paper we investigate the inverse optimization problem 
for polynomial optimization problems P as in i.e., in a rather general context 

which includes nonlinear and nonconvex optimization problems and in particular, 
0/1 and mixed integer nonlinear programs. Fortunately, in such a context, Puti- 
nar's Positivstellensatz [15, provides us with a very powerful certificate of global 
optimality that can be adapted to the actual computational capabilities for a given 
problem size. More precisely, and assuming y = (possibly after a change of vari- 
able x' = X — y), in the methodology that we propose, one computes the coefficients 
of a polynomial fd G R[x] of same degree do as / (or possibly larger degree if desired 
and/or possibly with some additional constraints), such that: 

• is a global minimizer of the related problem minx{/d(x) : x G K}, with 
a Putinar's certificate of optimality with degree bound d (to be explained 
later) . 

• fd minimizes ||/ — /||fe (where depending on k, \\ ■ \\k is the £i, £2 or £00- 
norm of the coefficient vector) over all polynomals / of degree do, having 
the previous property. 

Assuming K is compact (hence K C [—1,1]" possibly after a change of variable), 
it turns out that the optimal value pd '■= \\fd — f\\k also measures how close is /(O) 
to the global optimum /* of P, as we also obtain that /* < /(O) < /* + Pd if ^ = 1 
and similarly /* < /(O) < f* + Pd if fc = 00. 

In addition, for the ^i-norm we prove that fd has a simple canonical form^ namely 

n 

fd = f + W^ + J2Kxl 

i=l 

for some vector b e R", and nonnegative vector A G M", optimal solutions of 
a semidefinite program. (For 0/1 problems it further simplifies to fd = f + b'x 
only.) This canonical form is sparse as fd differs from / in at most 2n entries 
only (^ It illustrates the sparsity properties of optimal solutions of £1- 

norm minimization problems, already observed in other contexts (e.g., in some 
compressed sensing applications). 

Importantly, to compute fd, one has to solve a semidefinite program of size 
parametrized by d, where d is chosen so that the size of semidefinite program asso- 
ciated with Putinar's certificate (with degree bound d) is compatible with current 
semidefinite solvers available. (Of course, even if d is relatively small, one is still 
restricted to problems of relatively modest size.) Moreover, when K is compact, 
generically fd is an optimal solution of the "ideal inverse optimization problem" 
provided that d is sufficiently large! 

In addition, one may also consider several additional options: 

• Instead of looking for a polynomial / of same degree as /, one might allow 
polynomials of higher degree, and/or restrict certain coefficients of / to be the same 
as those of / (e.g. for structural modeling reasons). 

• One may restrict / to a certain class of functions, e.g., quadratic polynomials 
and even convex quadratic polynomials. In the latter important case and if the 
(7j's that define K are concave, the procedure to compute an optimal solution 
/(x) = b'x -I- x'Qx simplifies and reduces to solving separately a linear program 
(for computing b) and a semidefinite program (for computing Q). 
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The paper is organized as follows. In a first introductory section we present the 
notation, definitions, and the ideal inverse optimization problem. We then describe 
how a practical inverse optimization problem reduces to solving a semidefinite pro- 
gram and exhibit the canonical form of the optimal solution for the ^i-norm. We 
also provide additional results, e.g., an asymptotic analysis when the degree bound 
in Putinar's certificate increases and also the particular case where one searches for 
a convex candidate criterion. 



2. Notation, definitions and preliminaries 

2.1. Notation and definitions. Let R[x] (resp. M[x](i) denote the ring of real 
polynomials in the variables x = (xi,...,.t„) (resp. polynomials of degree at 
most d), whereas S[x] (resp. ^[xjd) denotes its subset of sums of squares (s.o.s.) 
polynomials (resp. of s.o.s. of degree at most 2d). For every a G N" the notation 
x" stands for the monomial x"^ ■ ■ ■ x""- and for every i G N, let :— {/? € N" : 
l3j < d} whose cardinal is s{d) = ("^'')- A polynomial / e ]R[x] is written 

X^/(X) = fo'^"^ 

and / can be identified with its vector of coefficients f = (/«) in the canonical basis 
(x"), a e N". Denote by 5* C M*^* the space of real symmetric matrices, and for 
any A e «S* the notation A ^ stands for A is positive semidefinite. For / e ]R[x]d, 
let 



' E iik = i, 



. max{|/„| : a e N^} if A; = oo. 

A real sequence z = (za), a G N", has a representing measure if there exists some 
finite Borel measure /x on such that 

Zc = [ x"d/i(x), VaeN". 

Given a real sequence z = (2;^) define the linear functional : M[x] — )• K by: 
/(=5]/„x") ^L,{f) = Y,faZa, /eM[x]. 

a a 

Moment matrix. The moment matrix associated with a sequence z = (za), a E 
N", is the real symmetric matrix Md(z) with rows and columns indexed by N^, and 
whose entry (a, (3) is just Za+p, for every a,P G N^. Alternatively, let V(i(x) € W^"^^ 
be the vector (x"), a e N^, and define the matrices (B^) c S^^'^^ by 

(2.1) Vd(x)vd(x)^ = E B,x«, VxeM". 

Then M<j(z) = Z^asNj^ 

If z has a representing measure /i then Md(z) ^ because 



(f,Md(z)f) = J fdii>o, Vf e: 
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Localizing matrix. With z as above and g e M[x] (with (/(x) — ^-^g-i^^), the 
localizing matrix associated with z and g is the real symmetric matrix Md(g z) with 
rows and columns indexed by I^^7 and whose entry (a, /?) is just (^^Zq,-!-^^^, for 
every a, P Ahernatively, let e be defined by: 

(2.2) g(x)v<j(x)vd(x)^ = V C„x", Vx e M". 



^2<i+doga 

Then Md(5z) Eogn- C^. 

If z has a representing measure /i whose support is contained in the set {x 
g(x) > 0} then Md(5z) ^ because 



(f,Md(5z)f) = y fgdfi>0, Vf e 



ps(d) 



With K as in (jl.ip . let go G IR[x] be the constant polynomial x i— > go(x) = 1, 
and for every j = 0, 1, . . . , m, let Vj :— [(deggj)/2] . 

Definition 2.1. With d,k eN andK as in flJ]) . /et Qfc(5) C R[x] and Qf. C M[x]d 
be the convex cones: 

(2.3) Q{g) := | a, g, : a, G S][x] j - 1, . . . , m I . 

(2.4) Qkig) := | ^ cr^ ffj : CTj G E[x]fe„„^ , j = l,...,m|. 

(2.5) Qiig) := Qfc(g) n M[x]rf 

FFe say i/iat every element h e Qkig) has a Putinar's certificate of nonnegativity 
on K, with degree bound k. 

The cone Q[g) is called the quadratic module associated with the g^'s. Obvi- 
ously, if /i e Q{g) the associated s.o.s. polynomials cr^'s provide a certificate of 
nonnegativity of h on K. The cone Q{g) is said to be Archimedean if and only if 

(2.6) X A/ - ||x|p G Q(5) for some M > 0. 

Let Psdd(K) C M[x]d be the convex cone of polynomials of degree at most d, 
nonnegative on K. The name "Putinar's certificate" is coming from the following 
Putinar's Positivstellensatz. 

Theorem 2.2 (Putinar's Positivstellensatz [TS]). Let K be as in U.l]) and assume 
that Q{g) is Archimedean. Then every polynomial f £ R[x] strictly positive on K 
belongs to Q{g). In addition, 

(2.7) cl [ Q Qt{g)\ = Psd^K), W e N. 

\fc=0 / 

The first statement is just Putinar's Positivstellensatz [TS] whereas the second 
statement is an easy consequence. Indeed let / G Psdd(K). If / > on K then 
/ G Qkig) for some k. If /(x) = for some x S K, let /„ := f + 1/n, so that 
/„ > on K for every n e N. But then /„ e ^k^^oQtig) ^^'^ result follows 
because ||/„ — /||i — )■ as n — oo. 

In fact, by results from Marshall [13] and more recently Nie [14], membership 
in Q{g) is also generic for polynomials that are only nonnegative on K. And 
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SO Putinar's Positivstellensatz is particularly useful to certify and enforce that a 
polynomial is nonnegative on K, and in particular the polynomial x i— > /(x) — /(y) 
for the inverse optimization problem associated with a feasible solution y e K. 

Notice that one may also be less demanding and ask y to be only a global e- 
niinimizer for some fixed e > 0. Again Putinar's Positivstellensatz is exactly what 
we need to certify (global) e-optimality by requiring /(•) — /(y) + e G Qk{g)- 

2.2. The ideal inverse problem. Let P be the global optimization problem /* = 
minx{/(x) : X e K} with K C K" as in and / S R[x]do where do := deg/. 

Identifying a polynomial / e IR[x]do with its vector of coefficients f = (/„) € 
^(do)^ one may and wiU identify K[x]do with the vector space K''('^o\ i.e., K[x]rfo 9 
/ O f e R^(*). Similarly, the convex cone Psddo(K) C ]R[x]d(, can be identified 
with the convex cone {h G W'^'^^'i : h o /i G Psddo(K)} of W'''^«\ So in the sequel, 
and unless if necessary, we will not distinguish between / and f . 

Next, let y G K and k G {l,2,oo} both fixed, and consider the following opti- 
mization problem V 

(2.8) V: min { ||/ - /|U : x /(x) - /(y) G Psd,„ (K) }. 

/eR[x]do 

Theorem 2.3. Let K C K." he with nonempty interior. Problem k2.l^) has an 

optimal solution f* G R[x](iQ. In addition, = if and only if y is an optimal 
solution ofP. 

Proof. Obviously the constant polynomial x i— ?> /(x) := 1 is a feasible solution with 
associated value i5 := \\f — f\\k- Moreover the optimal value of (|2.8p is bounded 
below by 0. Observe that || • ||fe defines a norm on M[x](jg. Consider a minimizing 
sequence (f^) C M[x]df,, j G N, hence such that ||/^ — — > p'^ as j — ?> oo. 
As we have Ij/-' — /|U < S tor every j, the sequence (/■') belongs to the £fc-ball 
Bfe(/) ■— {/ G I^Mdo ■ 11/ ~ /lU < <5}i a compact set. Therefore, there is an 
element /* G Bfc(/) and a subsequence (jt), t G N, such that J-'* /* as i oo. 
Let X G K be fixed arbitrary. Obviously (0 <)P(x) - P*(y) /*(x) - /*(y) 
as i oo, which implies /*(x) — /*(y) > 0, and so, as x G K was arbitrary, 
/* - /*(y) > on K, i.e., /* - /*(y) G Psddo(K). Finally, we also obtain the 
desired result 

p'^ = hm 11/^' - /lU = lim I1.P - /II, = llf - /IU-. 

J — ^oo I — ^oo 

Next, if y is an optimal solution of P then / := / is an optimal solution of V with 
value p'^ — 0. Conversely, if p*^ = then /* = /, and so by feasibility of /* (= /) 
for (|2.8p . /(x) > /(y) for all x G K, which shows that y is an optimal solution of 
P. □ 

Theorem 12.31 states that the ideal inverse optimization problem is well-defined. 
However, even though Psd^o (K) is a finite dimensional convex cone, it has no simple 
and tractable characterization to be used for practical computation. Therefore one 
needs an alternative and more tractable version of problem V. Fortunately, we 
next show that in the polynomial context such a formulation exists, thanks to the 
powerful Putinar's Positivstellensatz (Theorem 12.21 above) . 
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3. Main result 

As the ideal inverse problem is intractable, we here provide tractable formula- 
tions whose size depends on a parameter d £ N. If the polynomial /* in Theorem 
l2.3l belongs to Q{g) then when d increases the associated optimal value pjj converges 
in finitely many steps to the optimal value p'^ of the ideal problem (12. 8p . and /* 
can be obtained by solving finitely many semidefinite programs. And in fact this 
situation is generic. 

With no loss of generality, i.e., up to some change of variable x' = x — y we may 
and will assume that y = G K. 

3.1. A practical inverse problem. With d e N fixed, consider the following 
optimization problem P^;: 

Pd ■■ Pd-^ ^ min \\f-f\\k 

771 

(3.1) s.t. /(x) - /(O) = 5j(x), Vx e K" 

/ e K[x]do; cTj e S[x]rf_„^ , j = 0, 1, . . . , m, 

where do = deg/, and vj = [(degg-,)/2] , j = 1, . . . ,m. 

The parameter d e N impacts (|3.ip by the maximum degree allowed for the s.o.s. 
weights (cTj) C E[x] in Putinar's certificate for the polynomial x (-)• /(x) — /(O), 
and so the higher d is, the lower p^. Next, observe that in p.ip . the constraint 

m 

/(x) - /(O) = ^a,(x).9,(x), Vx e M", 

is equivalent to stating that /(x)-/(0) € with Qd°{9) as in Therefore, 

in particular, /(x) > /(O) for all x G K, and so is a global minimizer of / 
on K. So Pd is a strengthtening of V in that one has replaced the constraint 
/ - /(O) e Psdd„(K) with the stronger condition /- /(O) € Qd°i9)- ^nd so 
p*-' < for all d G N. However, as we next see, Pd is a tractable optimization 
problem with nice properties. Indeed, Pd is a convex optimization problem and 
even a semidefinite program. For instance, if fc = 1 one may rewrite Pd as: 

Pd 

A=>0,/,Zj 
S.t. 

(3.2) 



with Bq, as in (|2.ip and as in (|2.2p (with gj in lieu of g). If fc = oo then one 
may rewrite Pd as: 



"eN5^\{o} 

Aa+/a>/a, Va G N^^ \ {0} 

Aa - /« > -/a, Va G NS„ \ {0} 

fa, if < |a| < do 

J = l 

Z, ^0, j-0,l,...,m. 



(Zo,B„>+^(z„c^„; 



0, if a = or |a| > do 
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Pd 



(3.3) 



min A 

A>0,/.Z, 

S.t. A + /„>^, VaGNS„\{0} 
A-/a>-/a, VaeNS„\{0} 

/„, if < \a\ < do 



(Zo,B„)+^(Z„C^J = 
Zj ^ 0, j=0,l,...,m, 
and finally, if fc = 2 then one may rewrite as: 



0, if a = or lal > do 



(3.4) 



P2 '■= mm 

A>0,/,Zj 



s.t. 



aGNS^\{0} 

Aq /"a fa 

fa fa 1 



0, \faeNl\m 



(Zo,B„)+^(Z„C^j = 
ZjhO, j=0,l,...,m. 



/„, ifO< |a| < do 
0, if a = or \a\ > do 



Remark 3.1. Observe that in any feasible solution (/,A, Zj) in all formulations 
p.2p - p.4p . fo plays no role in the constraints of p.ip . but since we minimize ||/ — 
/II A; then it is always optimal to set /o = fo- That is, /d(0) = fo = fo = .f(0). 



Sparsity. The semidefinite program p.ip - p.2p has m + 1 Linear Matrix Inequal- 
ities (LMI's) Zj ^ of size 0{n'^), which limits its application to problems P of 
modest size. However large scale problems usually exhibit sparsity patterns which 
sometimes can be exploited. For instance, in we have provided a specialized 
"sparse" version of Theorem l2.2l for problems with structured sparsity as described 
in Waki et al. 117^. Hence, with this specialized version of Putinar's Positivstel- 
lensatz, one obtains a sparse positivity certificate which when substituted in (|3.1|) . 
would permit to solve for problems of much larger size. Typically, in 17j the 
authors have applied the "sparse semidefinite relaxations" to problem P with up to 
1000 variables! Moreover, the running intersection property that must satisfy the 
sparsity pattern for convergence guarantee of such relaxations is not needed 
in the present context of inverse optimization. This is because one imposes / to 
satisfy this specialized Putinar's Positivstellensatz. 



3.2. Duality. The semidefinite program dual of p.2p reads 



max fa{Ua - Va) {= L^{f{0) - f)) 

u,v>0,z ^ — ^ 

aGN5^\{0} 

S.t. u^+v^ < 1, VaeNS„\{0} 

Ua~v^ + z^ ^ 0, Va e \ {0} 

SaeNJ ^a'Ba ^ 
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which, recalhng the respective definitions ()2.1|) and ()2.2|) of the moment and local- 
izing matrix, is the same as 



(3.5) 



max 

u,v>0,z 

s.t. 



aeNS^\{0} 

ua + fa < 1, VaeNS„\{0} 
u^-Vc^ + Za, = 0, Va e \ {0} 
Md(z), Md-v^{gj z) ^ 0, j = 1, . . . ,m. 



Similarly, the semidefinite program dual of (13. 3p reads 



(3.6) 



max 

u,v>0,z 



s.t. 



E 



/a(Ua-«a) (- Lz(/(0)-/)) 
li„ + U„ < 1 



aeN5^\{0} 

E - 

aeN5^\{0} 

"a - fa + = 0, Va G N^^, \ {0} 



and the semidefinite program dual of 



(3.7) 



max 

Z,Aa 



S.t. 



E 



reads 
/„ 

fa -1 



1 



1 

1 



< 



1, VaeNS„\{0} 



+ Za 



0, VaeNl\{0} 



Md(z), Md_.,.(5jz) )^ 0, j = l,...,m 
A„)rO, VaeN3„\{0}. 

One may show that one may replace the criterion in p.7p with the equivalent 
concave criterion 



max I iz(/(0) -/))-! J2 [ 



Lemma 3.2. Assume that K C M" has nonempty interior. Then there is no 
duality gap between the semidefinite programs \3. 2\) and li3.5\) . h3.S\] and h3.(!\) . and 
{3.4^ and {3. ?[ ). Moreover, all semidefinite programs h3.2fl . 13. 3\} and |^.^[ ) have an 
optimal solution fd G M[x]rfQ. 

Proof. The proof is detailed for the case k = \ and omitted for the cases k = 2 and 
k = 00 because it is very similar. Observe that Pd> and the constant polynomial 
/(x) = for aU x e R", is an obviously feasible solution of (13. ip (hence of p.2p '). 
Therefore p]^ being finite, it suffices to prove that Slater's conditiorQ holds for the 



"^Slater's condition holds if there exists a strictly feasible solution, and so for the dual 113.51 1. if 
there exists z such that Mj{z), M^—uj {gj z) ^ 0, j = 1, . . . , m, and Ua +Va < 1, Va € N^^ \ {0}. 
Then from a standard result in convex optimization, there is no duality gap between I I3.2I I and 
l|3.5|l . and if the values are bounded then l|3.2|l has an optimal solution. 
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dual (|3.5p . Then the conclusion of Lemma 13.21 follows from a standard result of 
convex optimization. Let /i be the finite Borel measure defined by 



dx, 



BnK 



(where B is the usual Borel c-field), and let z = (z^), a G Nj^, with 
Za K / x" d^(x), a e N2 



for some k > sufficiently small to ensure that 



(3.8) 



x" d^(x) 



< 1, VaeN^,\{0}. 



Define Ua — max[0, — Zq] and Va — max[0, z^], a S NJJ^ \ {0}, so that Ua + Va < 
1, a G \ {0}. Hence {ua,Va,z) is a feasible solution of (|3.5p . In addition, 
M(i(z) )^ and 'M.d-v- [gj z) 0, j = 1, . . . , m, because K has nonempty interior, 
and so Slater's condition holds for p.5[) . the desired result. 

If /c = 00 one chooses z such that 



aeNS^\{0} 



x" dM(x) 



< 1, 



and if fc = 2 then one chooses z as in p.8p and 
a e N;J^^ \ {0}, such that 



1/2 Ka 



>- 0, for ah 



2k„ := K / x"dAi(x), VaeN5^\{0}. 



□ 

Theorem 3.3. Assume that K m (QTip /las nonempty interior, and let x* G K &e 

a global minimizer of P with optimal value /*, and let fd S I^Mdo he an optimal 
solution ofPd in iS.l]) with optimal value p^. Then: 

(a) G K is a global minimizer of the problem f^ = minx{/d(x) : x G K}. In 
particular, i/p^ = then fd = f cind is a global minimizer ofP. 

(b) If k = 1 then f* < /(O) < f* + Pd sup |(x*)"|. In particular, if 
KC [-1,1]" tften/* < /(O) < r 

(c) If k = 00 then f* < /(O) < f* + p'^ particular if K C 
[-1, 1]" then f* < /(O) < f* + s{do) pf. 

Proof, (a) Existence of fd is guaranteed by Lemma 13.21 From the constraints of 
(I31|) we have: /<j(x) - /(O) = Z^Jlo '^J W 5j (x) which implies that /^(x) > /(O) 
for all X G K, and so is a global minimizer of the optimization problem P' : 
minx{/d(x) : x G K}. 
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(b) Let X* e K be a global minimizer of P. Observe that with fc = 1, 

r=/(x*) = /(x*)-/d(x*) + /d(x*)-/d(0)+/rf(0) 

>o 

> /,(0)-|/,(x*)-/(x*)| 

> fM-\\fd^f\\i X sup |(x*)"| 

do 

(3.9) > /(O)-pi sup |(x*)"| 



since /d(0) = /(O); see Remark IXTI 

(c) The proof is similar to that of (b) using that with k — oo, 




□ 

So not only Theorem 13.31 states that is the global optimum of the optimiza- 
tion problem min{/rf(x) : x G K}, but it also states that the optimal value 
also measures how far is /(O) from the optimal value /* of the initial problem P. 
Moreover, observe that Theorem 13.31 merelv requires existence of a minimizer and 
nonemptyness of K. In particular, K may not be compact. 

3.3. A canonical form for the ^i-norm. When K is compact and if one uses 
the €i-norm then the optimal solution fd £ M.[x]do in Theorem 13.31 (with fc = 1) 
takes a particularly simple canonical form: 

As K is compact we may and will assume (possibly after some scaling) that 
K C [—1, 1]" and so in the definition of (jl.ip we may and will add the n redundant 
quadratic constraints (7„i+i(x) > 0, i = 1, . . . , n, with x i-^ g„i+i(x) = 1 — xf for 
every i, that is, 

(3.10) K = {x e R" : gj{-x) > 0, j = 1, . . . , m + n}, 

and 

{n+m I 
X! ^3 3j ■ CTj e S[x]d_„^ , j = 0, . . . , m + n i , 
j=o J 

which is obviously Archimedean. 

Theorem 3.4. Assume that K in iS.lOp has a nonempty interior and let G 
M[x](io be an optimal solution ofFd iS.l]) (with m-\-n instead ofm) with optimal 
value p\ for the £i-norm. Then: 
(i) fd is of the form 

■^'^ W - I ^(^) + b'x + ELi A* x| tf do > 1, 
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for some vector b £ M" and some nonnegative vector X* G E", optimal solution of 
the semidefinite program: 

n 

Pd—min l|b||i+^A, 

(3.12) ' 

s.t. /-/(0) + b'x + ^A.x^ eQrf(<?), A>0. 

i=l 

(ii) The vector b is of the form — V/(0) + X]jeJ(o) f'^'"' some nonnegative 

scalars {6j) (where j G J(0) if and only if gj{0) — 0). 

Proof, (i) Notice that the dual (|3.5I) ol p.2p is equivalent to: 
r max L,(/(0)-/)) 

(3.13) < s.t. Md(z), Mrf_„^(5jz) ^ 0, j = l,...,m + n, 
I |2a|<l, VaeN3^\{0}. 

Next, since WL^iz) >: one may invoke same arguments as those used in Lasserre 
and Netzer jl2i Lemma 4.1, 4.2], to obtain that for very a S Nj^j with |a| > 1, 

Md(z) ^ \za\< max {rRax[L^{xf),L:^{xf'^)]}. 

Moreover the constraint Md-i(5m+j z) ^ implies 'M.d-i{gm+iz){£,£) > for aU 
£, and so in particular, one obtains iz(x^''"^) > Lz(a;f'^) for ah fc = 1, . . . , d and 
ah i — 1, . . . ,n. Hence |zq| < maxi=i^..._„ Lz(a;f ) for every a S Nj,; with jaj > 1. 
Therefore in p.l3p one may replace the constraint \za\ < 1 for all a S N]j^^ \ {0} 
with the 2n inequality constraints ±Lz(xi) < 1, i = 1, . . . , n, if do = 1 and the 3n 
inequality constraints: 

(3.14) ± L^{x,) < 1, Lz(a;D <l,i = l,...,n 

if do > 1- Consequently, (j3.13p is the same as the semidefinite program 
r max Lz(/(0)-/)) 

(3.15) <^ s.t. Md{z),Md-y^{gjz) h 0, j = l,...,m + n, 
[ ±L^{x,) <l, L^ix^) <1, i = l,...,n. 

Let b^ — (bj) (resp. b^ — (6f )) be the nonnegative vector of dual variables associ- 
ated with the constraints L^ixi) < 1 (resp. —L^ixi) < 1), i = 1, . . . , n. Similarly, 
let Xi be the dual variable associated with the constraint Lz{xf) < 1. Then the 
dual of p.lSp is the semidefinite program: 



max V {{bl + hi) + A,) 

2—1 

n 

S.t. / - /(O) + (bl - b2)'x + ^ A, xl e QM 



bi,b2,A > 



which is equivalent to p.l2p . 

(ii) Let (b. A) be an optimal solution of p.l2p . so that 



m+n 

/ - /(O) + b'x + ^ A, = no + 51 9j, 

2=1 j = l 
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for some SOS polynomials aj. Evaluating at x = yields 

ao(0)=0; a,(0) <7j(0) = 0, j = l,...,m + n. 

Sj>0 

Differentiating and evaluating at x = and using that aj is SOS and (0)5^(0) — 0, 
j = 1, . . . , n + m, yields: 

V/(0) + b = ^a,(0)V5,(0) = ^^^5,(0), 

which is the desired result. □ 

From the proof of Theorem 13. 41 this special form of fd is specific to the £i-norm, 
which yields the constraint \za\ < 1, a € \ {0} in the dual p.5p and allows its 
simplification p.l4p thanks to a property of the moment matrix described in [12|. 
Observe that the canonical form (jS.lip of fd is sparse since fd differs from / in at 
most 2n entries only (recall that / has ("^'^") entries). This is another example 
of sparsity properties of optimal solutions of £i-norm minimization problems, al- 
ready observed in other contexts (e.g., in some compressed sensing applications). 
Moreover, it has the following consequence for nonlinear 0/1 programs. 

Corollary 3.5. Let K = {0, 1}", / G R[x]d and let y e K with h {i ■ Vi = 0} 
and I2 '■— {i '■ yi — 1}. Then an optimal solution fd G K[x]rf„ of the inverse problem 
hS.l]] for the £1 -norm, is of the form 

(3.16) /d(x) = /(x)+b^(x-y), 

for some coefficient vector b G R" such that bi > if i d Ii and bi < if i d I2, 
i = 1, . . . , n. Moreover, b is an optimal solution of the semidefinite program: 

b,A,7 ^ — ^ ^ — ^ 

n 

s.t. /(x) - /(y) + b^(x - y) = ao + ^(x? - x,) a, 

b, > 0, i e h; b, < 0, i e h; ctq G I][x]d; CF^ G R[x]d-i, i = 1, . . . , n. 

Proof. We briefly sketch the proof which is very similar to that of Theorem l3.4l even 
though K does not have a nonempty interior and c?o is not required to be even. Re- 
call that y G K could be assumed to be and so if K = {0, 1}" then the new feasible 
set after the change of variable u := x— y is now K = Ilig/iCiO' 1}) Ylieh ^J")- 
Similarly, let /' G M.[x:]d be the polynomial / in the new coordinates u, i.e., 
/'(u) /(u + y). So for every a G N", define a G {0, 1}" and C I2 by: 

ai := 1 if > and otherwise, i = 1, . . . ,n 
Aa :— {i I2 ■ < cti is even}, 

and let |A„| denotes the cardinality of A^. Then because of the boolean constraints 
uf = Ui, i G Ii and = — Ui, i G /2, in the definition of K, (j3.13p reads 

' max Lz(/'(0)-/') 

< s.t. Mrf(z) h 0; za^ (-l)IAJ a G N«rf 
ko|<l, yaeNl\{0}. 
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But this combined with Md(z) y imphes that M(i(z) can be simphfied to a 
smaller real symmetric matrix Mrf(z) with rows and columns indexed by only the 
square-free monomials x", a e ^2d- Indeed, every column a of Md(z) is exactly 
identical to ± the column of M(i(z) indexed by a. Also, invoking and with 
similar arguments, \za\ < max[maxig/j L^iui), max^g/^ — £z('"i)] for all a, and so 
p.l7l) is equivalent to: 

max £z(/'(0)-/') 

s.t. Md(z)^0; z„ = (-l)l'^°lza, aeN^<j 



(3.17) 



Lz(x,) < 1, i e /i 

-iz(x,) < 1, i(z h. 



Finally, let /i be a Borcl measure with support exactly K, and scaled to satisfy 
I / Uid^\ < 1, i = 1, . . . , n. Its associated vector of moment z = (J u"d/i), a e N", 
is feasible in p.l7p and Md(z) >~ 0. Hence Slater's condition holds for p.l7p . which 
in turn implies that there is no duality gap with its dual which reads: 



min > hi 



1=1 



s.t. /'(u) - /'(O) + V hu, - V hu, = ao(u) 



+ ^ CTi(u) (u- - Itj) + ^ (Tj(u) (m^ + Ui) 

ieii ieh 
b > 0; CTo e S[u]d; e R[u]d-i, i = l,...,n. 

Moreover, the dual has an optimal solution because the optimal value is bounded 
below by zero. Recalling that u = x — y and /'(u) = /(x), we retrieve the 
semidefinite program of the Corollary and so fd is indeed of the form (I3.16P . □ 

3.4. Structural constraints. It may happen that the initial criterion / e K[x] 
has some structure that one wishes to keep in the inverse problem. For instance, in 
MAXCUT problems on K = {— l,!}", / is a quadratic form x i— )■ x'Ax for some 
real symmetric matrix A associated with a graph (V, E), where Aij 7^ if and only 
if S E. Therefore, in the inverse optimization problem, one may wish that 
/ in p.ip is also a quadratic form associated with the same graph {V,E), so that 
/(x) = x'Ax with A,j = for aU (ij) ^ E. 

So if A/ C N^^ denotes the subset of (structural) multi-indices for which / and 
/ should have same coefficient, then in p.ip one includes the additional constraint 
fa = fa for all a S A/. Notice that G A/ because /o = /o! see Remark |3. II For 
instance, with K as in (|3.10p and k = 1, p.2p reads 



p\ :— mill 



"eNSo\{0} 

s.t. Xa+fa>fa.. Va € N^^ \ A/ 

(3.18) K~fa>~fa, ya&Nl\Af 



/a, ifaeA/\{0} 
(Zo,B„)+ ^(Z„C^J - <! /„, ifaeN^'„\A/ 

j=i { 0, if a = or la] > do 

'Z'j h 0, j — 0,1, . ■ ■ ,m + n, 
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and its dual lias the equivalent form, 

r max L,(/(0)-/)) 
(3.19) <^ s.t. Md{z),Md-y^{gjz) h 0, j = l,...,m + n, 

[ K\<1, VaGN3„\(A/U{0}). 

However now p.lSp may not have a feasible solution. In problems where do 
is even and A/ does not contain the monomials a € N^^ such that x" = xf, or 
x" — , i — 1, . . . , n, and if p.lSp has a feasible solution, then there is an optimal 
solution / with still the special form described in Theorem 13.41 but with 6fe = if 
a ^ Bk (z A.f (where all entries of vanish except the one at position k). 

3.5. Asymptotics v^rhen d — oo. We now relate P^, d £ N, with the ideal inverse 
problem V in (j2.8|) when d increases. 

Proposition 3.6. Let K be as in with nonempty interior. For every k — 

l,2,cxi, let fd G K[x]d(, (resp. f* G M[x]doj be an optimal solution of i3.1\) (resp. 
h2.^) ) with associated optimal value (resp. p^). 

The sequence {p^), d G N, is monotone nonincreasing and converges to p^ > p^. 
Moreover, every accumulation point f G R[x](io of the sequence {fa), d G N, is such 
that f - /(O) G Psddo(K) and \\f - f\\k = /5^ Finally, if f* - f*{Q) is m Q{g), 
then Pd — p'' = for some d. 

Proof. Observe that the sequence (/d), d G N, is contained in the ball {h : \\h — 
/lU ^ Pdoi *- l^Mdoi fo'" some do € N. So let / be an accumulation point of [fd)- 
Since fd~ f{0) > on K for all d, a simple continuity argument yields / — / (O) > 
on K, i.e., / — /(O) G Psdd(,(K). Moreover, the sequence (p^) is obviously monotone 
nonincreasing and bounded below by zero. Hence lim^-s-oo =: p'' > p'', and by 
continuity \\f - f\\k = p''. 

Finally, if /* - /(O) G Q{g) then /* - /(O) G Qd"{g) for some d, and so /* is a 
feasible solution of (j3.1l) but with value p^ < p^. Therefore, we conclude that /* is 
an optimal solution of p.ip . □ 

Proposition 13.61 relates Pd and p'^ in a strong sense when /* — /(O) G Q{g). 
However, we would like to know how restrictive is the constraint /* — /(O) G Q{g) 
compared to /* - /(O) G Psddo(K). Indeed, even though Psddo(K) = cl {i)f^f))Qe° 
when K satisfies the assumptions of Theorem 12.71 in general an approximating 
sequence {fi) C Q{g), ^ G N (with \\fe — f*\\k — > 0), does not have the property 
that /^(x) - fi{0) > for ah x on K. 

3.6. Q{g) versus PsddQ(K). Therefore the question is: How often a polynomial 
nonnegative on K (and with at least one zero in H.) is an element of Q{g)? This 
question can be answered in a number of cases which suggest that / > on K and 
f ^ Q{g) can be true in very specific cases only (at least when K is compact and 
Q{g) is Archimedean). Indeed / > on K implies / G Q{g) whenever: 

• / and —gj are convex, j = 1, . . . , n. Slater's condition holds and V^/(x*) >~ 
at the unique global minimizer x* G K; see e.g., de Klerk and Laurent 
i- 

• K C {0, 1}", i.e., for 0/1 polynomial programs, and more generally for all 
discrete polynomial optimization problems. 
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• Qid) is Archimedean, / has finitely many zeros in K and the Boundary 
Hessian Condition (BHC) holds at every zero of / in K; see e.g. Marshall 
[15] . That is, the BHC holds at a zero x* e K of / if there exists < fc < n, 
and some 1 < vi < ■ ■ ■ < Vk < m, such that: gy-^, . . . ^g^^, are part of a 
system of local parameters at x*, and the standard sufficient conditions 
for X* to be local minimizer of / on {x : g^. (x) > 0, j — 1, . . . , fc} hold. 
Equivalently, if (ti, . . . are local parameters at x* with tj — j — 
1, . . . , fc, then / can be written as a formal power series /o + /i + /2 • • ■ hi 
. . . , t„]], where each fj is an homogeneous form of degree j, 

/i=aiiiH hflfctfe with > 0, i = l,...,fc, 

and the quadratic form /2(0, . . . , 0, t}~+i, . . . , tn) is positive definite. 



k times 

For instance, if all the (finitely many) zeros x* of / are in the interior of 
K, one may take fc = and ii(x) = Xi — x* for all i. Then / e Q{g) if /2 
is positive definite. 

It turns out that under a technical condition on the polynomials that define K, 
the BHC holds generically, i.e., the set of polynomials / G M[x]d for which the 
BHC holds at every global minimizer on K, is dense in M[x]d; see Marshall [13j 
Corollary 4.5]. Finally, and in the same vein, a recent result of Nie [M] states that 
if the standard constraint qualification, strict complementarity and second-order 
sufficiency condition hold at every global minimizer of / on K, then f — f* € Q{g)- 
Moreover this property is also generic in the sense that it does not hold only if the 
coefficients of the polynomials / and gj, j — 1, . . . , m, satisfy a set of polynomial 
equations! In other words the three conditions hold in a Zariski open set; for more 
details see Nie [IH Theorem 1.1, Theorem 1.2 and §4.2]. 

So in view of Proposition 13.61 one may expect that limd_j.oo — p'' generically. 
On the other hand, given d € N, identifying whether = (or whether \p^ — p^\ < 
e for some given e > 0, fixed) is a open issue. For instance, p^ = p'^^^ (as is the 
case in Examples [2] and [3] below) is not a guarantee that p^ — p^ . 

We will also see in Section SjJlthat one may also approach as closely as desired an 
optimal solution of the ideal inverse problem (|2.8|) by asking y to be only a global 
e-minimizer (with e > fixed), provided that e is small enough. 

In addition, more can be said by looking at the dual of (|2.8p . 

The dual of the ideal inverse problem V. We now provide an explicit inter- 
pretation of the dual problems in p.5p - (|3.6p . Let A/(K) be the space of finite 
Borel measures on K. Then obviously (|3.5p is a relaxation of the following problem: 



which, denoting by i5o the Dirac measure at x = 0, and by P(K) the space of Borel 
probability measures on K, can be rewritten as 



(3.20) 





l{5o{I)-^{f)) 
l:t. ±7(^^(x") - <5o(x")) < 1, Va e 



^(K)=7- 
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Similarly, p.6|) is a relaxation of the following problem: 

(3.21) r°°= max J / (/(O) - /(x)) d^^lx) : ^ / x"d/i < 1 
uGM(K) /k" — he 

or, again, equivalently, 

max J 7 (Soif) K/)) : 7 E " ^ ^(K) = ^ 

Hence, in the dual problems p.20p and p.2ip one searches for a finite Borel measure 
/i which concentrates as much as possible on the set {x G K : /(x) < /(O)}, and 
such that its moments up to order do are not too far from those of a measure 
supported at {0} e K. 

In fact, and as one might have expected, p.20p (resp. p.2ip ) is the dual of V 
in ()2.8p with k — 1 (resp. with k = oo). For instance, with A; = 1, to see that weak 
duality holds, let / e K[x]do and /i e M(K) be an arbitrary feasible solution of 
(1211) and (|3?20| . respectively. Then: 

(/(O) -f)d^i = [ (/(O) - f)df, + [ if- /) d/i 



<0 



< El/" - /"I 



aeN5^\{o} 



/ x"d^(x) 



i.e., weak duality holds and ri < p^. We even have the following: 

Lemma 3.7. Let K in il.l]) be with nonempty interior and assume that Q{g) is 
Archimedean. Let p'' be as in i2. 8\) with k ~ l,oo, and let z'* ~ (2^) e M*'('') be a 
nearly optimal solution of iS. 5\) (or i3.13\) ). e.g., with value Lz;(/(0) — /) > p^ — l/d, 
for all d e N. 

//liminfzQ < co then lim — p^ and iS.2(J\) has an optimal solution p* G 

Af(K) which is supported on the set of global minimizers on K of the optimal 
solution f * £ M.['x]do of li2.8]) (which contains {0}). Hence either = in which 
case is an optimal solution ofP, or p^ > Q and f* has a another global minimizer 
i 7^ on K with f{x) < /(O). 

Proof. The proof for the case A: = 00 is omitted as very similar to that of the case 
k = I. Consider the subsequence c?i, i £ N, such that liminf Zg = lim z^' < 00. 

d — ^00 i — ^00 

Using the Archimedean property (|2.6p of Q{g), we proceed exactly as in the proof 
of Theorem 3.2 in [13 p. 57-59 ]. There is a infinite sequence z* = (z*), a G N", 
and a subsequence (still denoted di for notational convenience), such that for every 
a G N", z^' — )• z*. Moreover, from the convergence z'^* — )• z*, M.d{gjZ*) ^ for 
every c? G N and every j ~ 0,1, . . . ,m; hence by Putinar's Theorem 15|, z* is the 
sequence of moments of a finite Borel measure p* supported on K. Moreover, since 
|z^'| < 1 for all a G \ {0}, we obtain 

I / X" dp*\ = Kl = lim |z^-| < 1 VaeNlX {0}, 
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which proves that /i* is feasible for p.20p . Finally, by monotonicity of the sequence 
{p\), d e N, and using p\> > for all d, 

ri < pi < hm p\ = hm p\ = lim L,., (/(O) - /) 

a— )-oo i^oo i^oo 

= iz.(/(0)-/)= / {f{0)-f)df,*, 
Jk 

which proves that fi* is an optimal solution of (|3.20l) . and so ri — p^. 
Finally, since r(0) = /(0), 

/ ifio)-f)dp* = I irio)-hdp*+ 1 {r-f)dp* 

Jk Jk Jk 

^ ■' 

<o 

< \\r-f\\i = p\ 

which implies /i*({x : /*(x) — /*(0) > 0}) = 0, that is, the support of p.* is 
contained in the set of global minimizers of /* (which contains {0}). Therefore, if 
p^ > then necessarily there is another global minimizer 7^ x G K of /* with 
/(i) < /(O), otherwise p^ = J (/(O) - f)dp* =0. □ 

3.7. Convexity. One may wish to restrict to search for convex polynomials / S 
M[x]jiQ (no matter if / itself is convex). For instance if the gj^s are concave (so 
that K is convex) but / is not, one may wish to find the convex optimization 
problem whose y e K is an optimal solution and with convex polynomial criterion 
/ e IR[x](j(, closest to /. 

If do > 2 then in (|3.ip it suffices to add the additional Putinar's certificate 

m 

(3.22) (x,u) ^ u^V2/(x)u - ;^V^j(x,u).g,(x) + 7/^„+i(x,u)(l- ||u||2), 

i=o 

with t/jrn+i G IR-[x, u] and ipj G Sd-uj [x, u], for all j = 0, 1, . . . , m. Indeed, p.22p 
is a Putinar's certificate of convexity for / on K, with degree bound d. As the 
coefficients of the polynomial (x, u) 1— u^V^/(x)u are linear in the coefficients of 
/, p.22p will translate into additional semidefinite constraints in p. 21) . 

If do < 2, i.e. if /(x) ~ ^x^Ax + b-^x + c for some real symmetric matrix A e 
^nxn^ some vector b e M" and some scalar c G R, then /(x) = ix'^Ax + b-^x + c 
for some real symmetric matrix A € R"^", some b £ M" and some c e M. In 
that case, in p.ip it suffices to add constraint V^/(x) — A ^ 0, which is just a 
Linear Matrix Inequality (LMI). And therefore, again, p.ip can be rewritten as a 
semidefinite program, namely (|3.2|) - ()3.4p with the additional LMI constraint A ^ 0. 

Notice that for fc = 1, 2, it also makes sense to search for / e M[x]2 even if / has 
degree dp > 2, i.e., if /(x) — c + h^x + ^x^Ax + h{x) where h £ M[x] does not 
contains monomials of degree smaller than 3. This means that one searches for the 
convex program with quadratic cost closest to /. 

So for instance, in the case where one searches for / e M[x]2, and given y G K 
let J(y) := {j £ {1, . . . , m} : gj{y) = 0} be the set of constraints that are active 
at y. If the g^'s that define K are concave then one may simplify Writing 
/ — ix^Ax + b^x + c, and with k — 1,2, p.ip now reads: 
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:= min ||/ - /||fc 

A,b,A 

s.t. Ay + b = ^J^9j{y) 

AhO;Xj>0, jeJ(y). 

So, as we did in the previous section, and possibly after the change of variable 
x' :~ X — y, with no loss of generality one may and will assume that y = 0, in 
which case (|3.23p simplifies to 

/ := min ||/ - /||fe 

A,b,A 

(3.23) s.t. b = ^ A, V9^(0) 

je./(0) 

A ^ 0; Xj > 0, j e J(0), 

which in turn simplifies to 

pi = min||A-A||i+min||b- ^ A,Vg^-(0)||i 

(3.24) 

p°° = sup 

Observe that (I3.24p can be solved in two steps. One first solves the problem 
minA>o l|t> — X]jej(o) AjV(7j(0)|jfc, which is a linear program with finite value, hence 
with an optimal solution. One next solves the problem min^v^Q ||A — A||fe which 
computes the ^fc-projection of A onto the closed convex cone of positive semidefinite 
matrices (a semidefinite program with an optimal solution). 

Lemma 3.8. Let K C K" be as in il.ll) with g.j being concave for every j = 
l,...,m. Then \3.2S]) has an optimal solution f* G K[x]2 and is an optimal 
solution of the convex optimization problem P' : min{/*(x) : x G K}. 

Proof. Let (/, A) (with / e K.[x]2) be any feasible solution of p.23p . The constraint 
in (|3.23p states that VL(0) = 0, where L e M[x] is the Lagrangian polynomial 
X n- i(x) := /(x) — X]je/(o) Aj5j(x), which is convex on K because the gj^s are 
concave, the Aj's are nonnegative, and / is convex. Therefore VL(0) = implies 
that is a global minimizer of L on M" and a global minimizer of / on K because 

(3.25) /(x) > i(x) > i(0) = /(O), Vx e K. 

It remains to prove that p.23p has an optimal solution /*. But we have seen that 
p.23p is equivalent to (|3.24l) for which an optimal solution can be found by solving 
a linear program and a semidefinite program. □ 

So in this case where the g^-'s are concave (hence K is convex), one obtains the 
convex programming problem with quadratic cost, whose criterion is the closest to 
/ for the ^fe-norm. 



A>-0 ^>0 



mmilA- A||oo , min||b- }^ AjV5j(0)|| 
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4. Global e-opxiMALixY 

One may be less demanding and ask y G K (or G K after a change of variable) 
to be only a global e-minimizer. That is, one searches for a polynomial / € R[x](io 
as close as possible to / and such that /(x) — /(O) > — e for all x S K and some 
e > fixed. Then we will see that one may approach as closely as desired an 
optimal solution of the ideal inverse problem (|2.8p . With K C [—1, 1]" as in (|3.10p . 
the analogue of Problem (|3.ip reads: 

Pde — ^ min 11/ -/life 
/,<TjeR[x] 

m+n 

(4-1) s.t. /(x) - /(O) + e = E ^^■(^) ffj- W' e ^" 

/ e K[x]rfo; cTj e S[x]d-t,^., j = 0,1,..., m + n. 
For instance, with A: = 1 (13.21) now reads 



:= min ^ 

X>0,f,Zj 



min 2^ Xa 



(4.3) 



aeN5^\{0} 

S.t. A„ + /„>/„, VaeNS„\{0} 
(4.2) A„ VaGNS„\{0} 

m+n ( /„, if < |a| < do 

(Zo,B„)+ ^(Z„CJj = <^ 0, if |a| >do 

j=i [ e, if a = 

ZjhO, j = 0,1,..., m + n. 

while its dual reads 

(pij* = max -e2;o+ V /«(«„ -«„)(= iz(/(0) -/- e)) 

u,v>0,z -"^^ — ' 

aeN5^\{0} 

s.t. Ua + vo < 1, VaGN3„\{0} 

Ma - + 2a = 0, Va G \ {0} 

Md(z), Md_„^.(gj z) ^ 0, j = 1, . . . ,m + n. 

Again with no loss of generality and possibly after a change of variable, we assume 
that y = G K. 

Lemma 4.1. Let K be as in iS.lO]) and let he the optimal value of the ideal 
inverse problem V in V2.8\] . Then Jar every fixed e > there exists G N such 
that p^^ < p^ for all d> d^. 

Proof. Let /* G K[x]do be an optimal solution of the ideal inverse problem V with 
value p'' = ||/-/*||fc. Observe that the polynomial X i-^- /*(x)-/*(0) + e is strictly 
positive on K and so by Theorem 12 .21 it belongs to Q{g)\ and so it belongs to Qd{g) 
as soon as d > de (for some de G N). Hence /* is a feasible solution of (|4.ip which 
implies the desired result p'^ > for all d > . □ 

The following analogue of Theorem 13 . 31 shows that the optimal value of (|4.ip 
is still helpful to bound the quantity /(O) — /* (where /* is the global optimum of 
problem P). 

Theorem 4.2. Assume that K in iS.lO]) has nonempty interior and let x* G K 6e a 

global minimizer o/P with optimal value /*. For every e > fixed, let fde G K[x]do 
be an optimal solution of (^TTP ^^th optimal value p^^. Then: 
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(a) G K zs a global e-minimizer of the problem f^^ = minx{/dc(x) : x G K}. 
In particular, if p^^ — then fde — f and is a global e-minimizer ofF. 

(b) // fc = 1 then f* < /(O) < /* + e + pi^. 

(c) Ifk = ^thenf* < /(O) < f* +e + s{do)pT,- 

The proof is omitted as being a verbatim copy of that of Theorem 13.31 and using 
/* - /(x*) - /(x*) - /,(x*) + /rf(x*) - /d(0) +/d(0). 

" V ' " v ' 

Concerning the canonical form associated with the £i-norm we also have the fol- 
lowing analogue of Theorem 13.41 

Theorem 4.3. Assume that K in iS. 1 0|) has a nonempty interior and for every 
e > 0, let fde G K[x]do be an optimal solution of j[ j with optimal value for 
the £i -norm. Then fde is of the form: 

n 

(4.4) /,,(x) = /(x)+b'x + ^A:a;?, 

1=1 

for some vector b G R" and some nonnegative vectors A* € M", optimal solution of 
the semidefinite program: 

n 

pL := llb||i+^Ai 

s.t. /-/(0) + b'x+ ^ A,x2 + eeQ,(g), A > 0. 
1=1" 

(AndX* =Oifdo = 1.) 

We end up with the analogue of Proposition [311] for the asymptotics as d — )■ cx3. 
For every e > 0, let us call Ve the analogue of problem V (= Vo), i.e., 

(4.5) Ve-. ^ min { \\f - f\\k : x ^ /(x) - /(y) + e G Psd<i„(K) }. 

/eR[x]do 

Proposition 4.4. For every e > fixed, Problem |^.5[ ) /las an optimal solution 
f * G M[x]c;„, and = i/ and onZy i/y is a global e-minimizer of P. 

The proof is similar to that of Theorem 12.81 Next, interestingly, we are able to 
relate (|4.ip and the ideal inverse problem (|2.8p as e 0. 

Proposition 4.5. Lei K in iS.lO]) be with nonempty interior. Let y = G K and 

p^ be the optimal value of the ideal inverse problem V in Ii2. 8\) and let fde G K[x]d(, 
(resp. f* G R[x](iQj be any optimal solution of (resp. ^4.5^ ). 

(i) Let > 0, £ G N, be such that ££ — )■ as i oo. Then every accumulation 
point f G M[x]c;q of the sequence {f*J C M[x]dn, ^ G N, zs an optimal solution of 
the ideal inverse problem I12.8\) . 

(ii) If for every > 0, d^ G N is sufficiently large, every accumulation point 
of the sequence (fdeet) C R[x]do is an optimal solution of the ideal inverse problem 
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Proof, (i) As II/,* - /life < p'' for all £, the sequence (/* ) C M[x]rf„, £ e E, has 
accumulation points. So consider an arbitrary converging subsequence (still denoted 
(/*J for notational convenience) /,* — > / G ]R[x]d(,, as £ -> oo. 

Observe that /* (x) - /,* (0) + > for all x G K and all £eN. With x e K 
fixed, arbitrary, letting ^ oo yields /(x) — /(O) > 0, and so x /(x) — /(O) G 
Psddo(K). Moreover, p^^ < p'' for all £ yields ||/ — / ||fe < p'^, which in turn implies 
that / is an optimal solution of the inverse problem (|2.8p . As the accumulation 
point / was arbitrary the result follows. The proof of (ii) is similar if one recalls 
that by Lemma HTTl p^,^ = ||/ — fdeeWk < P*^ ^oi all £ and all d sufficiently large, 
say d > d£. □ 

Hence, by asking y (= 0) G K to be only a global e-minimizcr, one may obtain 
a polynomial fde G KIxJ^q as close as desired to an optimal solution of the ideal 
inverse problem provided that e is sufficiently small and d is sufficiently large. 

4.1. Illustrative examples and discussion. We here provide some simple illus- 
trative examples and show that the representation of the set K may be important 
for getting a Putinar certificate faster. 

Example 1. Let n = 2 and consider the optimization problem P : /* = minx{/(x) : 
X G K} with X ./(x) = xi + X2, and 

K = {x G : xiX2 > 1; 1/2 < x < 2 }. 

The polynomial / is convex and the set K is convex as well, but the polynomials 
that define K are not all concave. That is, P is a convex optimization problem, 
but not a convex programming problem. The point y=(l,l)GKisa global 
minimizer and the KKT conditions at y are satisfied with A = (1, 0, 0) G M^, i.e., 

V/(x) - AiV5i(x) = with X = (1, 1) and Ai = 1. 

However, the Lagrangian 

X L(x) := /(x) - /* - Ai gi{x) = xi + X2 - I ~ xiX2, 

is not convex and (1, 1) is not a global minimizer of L on M^. This example just 
illustrates the fact that even in the convex case where the g^'s are not concave, the 
KKT conditions do not provide a certificate of global optimality, contrary to "convex 
programming" where since L is now convex, obviously using L(x.) > L(y) — 
(because VL(y) = 0), 

/(x) - /* > L(x) > L{y) - 0, 

whenever x G K, and so /(x) > /* for all x G K, the desired certificate of global 
optimality. 

Next, if we now use the test of inverse optimality with d — 1^ one searches for a 
polynomial fd of degree at most do = 1 7 and such that 

2 

/d(x) - /d(l, 1) = (7o(x)+ai(x)(xia;2-l) + ^^,(x)(2-a;,) + 0»(x)(a;, -1/2), 

i=l 

for some s.o.s. polynomials ai,ipi (pi G S[x]o and some s.o.s. polynomial cto G S[x]i. 
But then necessarily cti = and ipi, (f>i are constant, which in turn implies that ctq 
is a constant polynomial. A straightforward calculation shows that /i(x) = for 
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all X, and so p\ — 2. And indeed this is confirmed when solvingj p.5|) with d ^ 1. 
Solving again p.Sp with now d = 2 yields P2 — 2 (no improvement) and with d = 3 
we obtain the desired result = 0. 

On the other hand, if now K has the representation: 

{x : a;ia;2 - 1 > 0; (a;, - l/2)(2 - x,) > 0; i = l,2}, 

then the situation differs because in fact 

1 2 4 2 ^ 

XI+X2-2 = - + -{Xi - X2f + -{xiX2 - 1) + 5 Y^i^^ - V2)(2 - Xi), 

i—l 

i.e., f — f* has a Putinar's certificate with degree bound d = 1. Hence the test of 
inverse optimality yields p\ = with fi = f. 

The above example illustrates that the representation of K may be important. 

Example 2. Again consider Example [T] but now with y=(l.l,l/l.l)eK, which 
is not a global optimum of / on K any more. By solving p.Sp with d = 1 we 
still find pI = 2 (i.e., /i = 0), and with d = 2 wc find p2 « 0.1734 and /2(x) « 
0.8266 a;i + X2- And indeed by solving (using GloptiPoly) the new optimization 
problem with criterion /2 we find the global minimizer (1.1,0.9091) ~ y. With 
d = 3 we obtains the same value pi = 0.1734, suggesting (but with no guarantee) 
that /2 is already an optimal solution of the ideal inverse problem. 

Example 3. Consider now the disconnected set K := {x : X1X2 > 1; x^ + 
X2 <~ 3} and the non convex criterion x 1— )■ /(x) := —xi — a;| for which x* = 
(-0.618,-1/0.618) e 9K is the unique global minimizer. Let y ;= (-0.63,-1/0.63) G 
9K for which the constraint X1X2 > 1 is active. At steps d = 2 and d = 3 one 
finds that fd = and p^ — That is, y is a global minimizer of the trivial 

criterion/(x) = for all x, and cannot be a global minimizer of some non trivial 
polynomial criterion. 

Now let y — (—0.63, — \/3 — 0.63^) so that the constraint xf +x\ <~ 3 is active. 
With obtain p\ = ||/||i and A = 0. With d = 2 we obtain = 1.26x1 - x^. With 
d = 3 we obtain the same result, suggesting (but with no guarantee) that /2 is 
already an optimal solution of the ideal inverse optimization problem. 

Example 4. Consider the MAXCUT problem max{x'Ax : x^ = 1, z = 1, . . . , n} 

where A = A' G M"^" and Aij — 1/2 for all i ^ j. For n odd, an optimal solution 
is y = (uj) with j/j = 1, j = 1, . . . \n/2~\ , and y.j = —1 otherwise. However, the first 
semidefinite relaxation 

n 

max {A : x'Ax - A = cr + ^ 7^(0;^ - 1); (t G S[x]i; A,7 G M} 

provides the lower bound — n/2 (with famous Goemans- Williamson ratio guaran- 
tee). So y cannot be obtained from the first semidefinite relaxation even though 

it is an optimal solution. The inverse optimization problem reads; Find the qua- 
rt 

dratic form x 1— > x'Ax such that x'Ax — y'Ay = ct + 7^ {xf — 1), for some 



■^To solve 113.51 1 we have used the GloptiPoly software of Henrion et al. [5], and dedicated to 
solving the Generalized Problem of Moments whose problem l|3.5p is only a special case. 
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(T e S[x]i, A, 7 G M, and which minimizes the ^i-norm ||A — A||i. This is an in- 
verse optimization problem with structural constraints as described in Section 13.41 
(since we search for a quadratic form and not an arbitrary quadratic polynomial 
/2). Hence, solving (|3.18|) for n = 5 with y as above, we find that 

2/3 2/3 1 1 ' 
2/3 2/3 1 1 
2/3 2/3 1 1, 

1 1 10 1 
1 1 110 

that is, only the entries € {(1,2), (1,3), (2,3)} are modified from 1/2 to 1/3. 

Conclusion 

We have presented a paradigm for inverse polynomial optimization. Crucial is 
Putinar's Positivstellensatz which provides us with the desired certificate of global 
optimality for a given feasible point y G K and a candidate criterion /. In addition, 
to some extent, the size of the certificate can be adapted to the computational ca- 
pabilities available. Finally, and remarkably, when using the €i-norm the resulting 
inverse optimal criterion / has a simple and explicit canonical form. We hope that 
the concept of inverse optimization will receive more attention from the optimiza- 
tion community as it could even provide an alternative stopping criterion at the 
current iterate y G K of any local optimization algorithm for solving the original 
problem P. 
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